• 专利附录
  • 专利说明
  • 权利要求
  • 类似技术
  • 同族专利
  • 引用文献
  • 法律状态
  • 优先权
    • 专利摘要:
    The invention relates to a remote diagnostic assistance device during a verification of the state of an aircraft engine made in situ by at least one operator by means of an endoscope used to capture images of said engine and perform measurements to be transmitted to at least one remote terminal used by at least one remote expert to perform a collaborative analysis of said images and said measurements. The device according to the invention further comprises a portable housing controlled by the operator and provided with means for wireless communication with the endoscope, and means enabling the operator to manually and / or automatically enrich the captured images and measurements made with audio and / or video and / or text comments.
    公开号:FR3033046A1
    申请号:FR1551523
    申请日:2015-02-23
    公开日:2016-08-26
    发明作者:Florie Remond；Erwan Guerin；Jade Guyenne；Bernard Lhermenier；Sylvain Squedin
    申请人:SNECMA SAS；
    IPC主号:
    专利说明:

    [0001] TECHNICAL FIELD The invention lies in the field of the control of aircraft engines under development or under maintenance after landing. to a flight, or scheduled maintenance, and more specifically relates to a device and a method of remote diagnostic assistance during an audit of the state of an aircraft engine performed in situ by an operator using an endoscope or a camera used to capture images of said engine and to perform measurements to be transmitted, with documentation and reference images, to at least one remote terminal (10) used by at least one least one remote expert to perform a collaborative analysis of said images and said measurements. The invention also relates to a remote diagnostic assistance method during an endoscopy of an aircraft engine. The invention further relates to a computer program stored on a recording medium and having instructions for performing the steps of the method. STATE OF THE PRIOR ART When verifying the state of an aircraft engine after a landing following a flight, in scheduled maintenance or during propulsion unit development tests or aircraft engine component tests , the engine parts are checked and tested in situ by operators at the foot of the wing of the aircraft or in development, test or maintenance facilities. These engines can sometimes be dismantled for expert appraisal by technical experts who are not always located at the test and test site and must travel to see the condition of the components and provide technical advice and guidance to the operators.
    [0002] 3033046 2 Decisions must be made quickly and unambiguously in order to decide on levels of damage and allow a quick return to operation of the aircraft, or to complete the test campaign. Currently technical decisions involving multiple sites and 5 remote people are made via many phone conversations or emails that may include photos or videos. This procedure involves repeated iterations when there is a misunderstanding between the operators who carry out the tests and the experts who analyze the results of these tests. This forces the operators to take a very large number of shots to avoid missing a critical area of the engine controlled. In aviation, for imperative reasons of safety, the sharing of technical information must be unambiguous. For this purpose, remote experts must have measurements, images and videos made according to specific angles of view.
    [0003] The operators at the foot of the installation must therefore be able to move easily to easily access the different areas to control. This is not always possible with the current devices because of the space requirement due to the use of several tools connected by several cables which are an inconvenience for the operators at the foot of the devices to be tested, and the fact that both hands operators often taken in space, very confined, in which they operate when they are under the wing (a few tens of centimeters of recoil only and). Furthermore, if the communication conditions are not optimal (presence of disturbances, lack of network coverage, electromagnetic disturbances), the images and measurements made may be of insufficient quality to enable remote experts to make a reliable diagnosis. An object of the invention is to allow the operators who perform the measurements to access without constraints to all areas of the engine to be controlled and to be able to move while performing inspection, measurements, and their sharing, etc. in real time with remote experts.
    [0004] Another object of the invention is to provide the experts with accurate information, in particular, measurements taken and images taken in key areas to establish a diagnosis whatever the weather conditions and whatever the topology of the place where the tests are carried out.
    [0005] SUMMARY OF THE INVENTION This object is achieved by means of a compact, space-saving device adapted to all types of endoscopic equipment and analog cameras, and to all contexts of endoscopies or videos of propulsion systems or components. (under the wing, deposited, etc.) so as to allow operators located on the control site and 10 tests to reach, without constraints, the different zones of the propulsion unit under test to identify and to quickly visualize possible faults and to communicate in real time the captured images and the results of measurements. This device comprises at least one endoscope equipped with a camera used in situ by at least one operator to capture images of said engine and to make measurements intended to be transmitted to at least one remote terminal used by at least one remote expert in order to to perform a collaborative analysis of said images and said measurements. The device according to the invention further comprises a portable box controlled by the operator located at the control and testing site provided with means for communicating wirelessly with the endoscope and means enabling said operator to enrich manually or automatically captured images and measurements made with audio and / or video and / or text comments. With added audio and / or video and / or text comments, the remote experts can guide the localized operator to the control and test site during testing and can assist in making decisions and considering corrective actions in a very short time. Said housing further comprises a wireless communication interface enabling the operators located on the control and testing site to exchange with the expert, in real time and in an interactive manner, the captured images, the documentation and the data. the results of the measurements carried out enriched by the audio and / or video and / or text comments. The box further comprises a connection to be connected to an audio headset, a tablet and / or a laptop and a memory for recording the information and images captured and the measurements exchanged and a conversion module captured images of a data format specific to the endoscope to a data format exploitable by the remote terminal. The remote diagnostic aid method according to the invention comprises the following steps: transmitting said images and said measurements of the endoscope to a housing controlled by the first operator; - exchange in real time and interactively, via a wired or wireless connection, the images, the enriched measurements and documentation between the portable box and at least one remote terminal used by at least one expert to analyze said images and said measures and establish a collaborative diagnosis. to manually and / or automatically enrich the images and measurements received by the portable box with audio and / or video and / or text comments, This method furthermore comprises a conversion of the results of the measurements made and the captured images of a data format specific to the endoscope to a data format that can be used by the portable box and the remote terminal. The method according to the invention is implemented by means of a computer program stored on a recording medium and comprising instructions for carrying out the steps of said method when it is executed on a tablet or a computer.
    [0006] BRIEF DESCRIPTION OF THE DRAWINGS Other characteristics and advantages of the invention will emerge from the description which follows, taken by way of non-limiting example, with reference to the appended figures in which: FIG. 1 schematically represents a device according to FIG. 2 schematically represents the steps of a method according to the invention.
    [0007] DETAILED DESCRIPTION OF PARTICULAR EMBODIMENTS FIG. 1 diagrammatically represents a device for remote diagnostic assistance during a verification of the state of an aircraft engine, for example an endoscopy, carried out in situ by operators 20 at the foot of the wing of the aircraft 5 or in development, testing or maintenance facilities for capturing images of the engine of the aircraft and for capturing measurements intended to be transmitted to several remote terminals 22 used by several experts 24 to perform a collaborative analysis of said images and said measurements. The images and measurements are made by means of an endoscope 26 or a camera 28 which communicate via a wireless link with a portable housing 30 which can be connected to a portable tablet or laptop controlled by the operators. The tablet 30 includes software enabling the operators 20 to enrich the captured images and the measurements made with audio and / or video and / or text comments. These comments can be entered manually by means of a virtual keyboard or a voice command 15 and / or automatically from a Near Field Communication (NFC) reader or an inertial unit associated with the endoscope. in addition, said tablet 30 is equipped with a wireless communication interface which provides a connection to the Internet through an antenna 32 arranged on the site of the facilities. The tablet 30 further comprises a connectivity test module and a bandwidth calculation module for checking, respectively, whether the network allows for reliable communication between the operators 20 and the experts 24, and for estimating the maximum bit rate. between the tablet 30 and the remote terminals 22. In order to maintain maximum image quality and minimum latency, the tablet 30 also includes an encoder for adjusting the bit rate as a function of the network characteristics and parameters. encoding determined by an acquisition card which, thanks to its integration in the housing of the device, can be associated with the endoscope 26 such as for example the resolution (ex: 1024x768), the number of frames per second (ex: 24fps ) and the type of encoder (eg H264, VP8). These encoding parameters are transmitted to the tablet 30 to allow the encoder to adjust the encoding of the video images according to the selected network.
    [0008] In a preferred embodiment, the tablet 30 comprises a module, hereinafter called context generator, intended to generate an operating context based on information provided by a GPS, weather data in relation to the location of the device. endoscopy obtained via APIs (Application Programming Interface), airport data (eg altitude, etc.) and data on the communications network (type of network, flow, jitter, latency, etc.). FIG. 2 illustrates the steps of a test and collaborative analysis session performed using the device of the invention. Before performing the endoscopy, in step 40, the operator 20 initiates an available connectivity and network search test. During this step, the connectivity test module calculates the bandwidth and selects, in step 42, the most appropriate bit rate to perform reliable exchanges between the tablet 30 and the remote terminals 22. At step 44 the operator 20 makes measurements and video image captures on the areas of the engine to be tested. In step 46, the encoder of the tablet 30 encapsulates the video packets in a transport protocol (eg RTP, RTMP) to publish the video stream via the internet network to a video or image sharing server and /or his. Note that from the beginning of the video stream publication and throughout the endoscopy the context generator extracts the information describing the context of the endoscopy (step 48). These are then recorded for a restitution in the final report. The video capture system of the extracted endoscope (step 50) an image on twenty-four per second (24fps). This image is processed by a fault finding software module installed in the tablet 30 and dedicated, at the same time, to the detection of anomalies or to the recognition of QR code or any other marking system of the engine parts as well as temporal information (timestamp) relating to the publication of the measurements made and the captured images. In step 52, the extracted image or part of this image is compared by the fault finding software module to anomaly models previously stored in a first database 54. If an anomaly is detected or if an uncertainty threshold is reached, the fault search module generates notifications which are displayed on the screen of the tablet 30 and on the screens of the remote terminals 22. The operator 20 enriches these notifications with audio comments and / or video which are transmitted in real time (step 56) to the remote experts 24. Further information is automatically added to the images captured from a Near Field Communication (NFC) reader and / or an inertial unit. associated with the endoscope 26. In step 58, the fault finding module reads the QR code or any other visual marking system of the part, consults a second database 60 10 including information relating to all engine parts and their respective locations and functions in the propulsion unit. Preferably, the location of the part in the propulsion unit uses three potential sources of information: Central inertial, embedded on the endoscopic camera, which provides displacement information and which uses the information of the engine planes stored in a third base. 61 to estimate the position of the camera. This information is automatically added to the captured images. The location information provided at step 58 from the recognition of visual tags; The information obtained by reading NFC tag (or similar) by means of a reader mounted on the head of the camera. In step 62, the location, detection / recognition of parts or anomalies data obtained in the previous steps are added automatically and in real time to the video stream for transmission to the experts 24. Step 64. In step 66, the data produced during the session are aggregated on a time line, thus allowing replay of the video, retrieval of the documentation or reference images with all the data of the endoscopic observation.
    [0009] If the experts believe that the information received in real time is sufficient to establish a diagnosis, they transmit their diagnosis directly to the operator 20 orally or via an electronic mail. At the end of the inspection, the shared session is interrupted and the data (video photos, conversations, etc.) exchanged during the session are stored (step 70) in a memory 72. The device according to the invention offers a hardware and software solution that relies on standardized interfaces and adapts to all types of endoscopic equipment and cameras (and not to a single make and model) and to all endoscopic or propulsion set videos or components. 10
    权利要求:
    Claims (7)
    [0001]
    REVENDICATIONS1. Device for remote diagnostic assistance during an aircraft engine condition check carried out in situ by at least one operator (20) by means of an endoscope (26) or a camera used to capture images of said engine and measurements to be transmitted to at least one remote terminal (22) used by at least one remote expert (24) for collaborative analysis of said images and measurements, this device is characterized in that it further comprises a portable housing (30) connectable to a screen and controlled by the operator (20) and provided with means for wireless communication with the endoscope (26) and means allowing the operator (20) to manually and / or automatically enrich the captured images and the measurements made with audio and / or video and / or text comments.
    [0002]
    2. Device according to claim 1 wherein said portable housing (30) further comprises a wireless communication interface allowing the operator (20) to exchange with the remote experts (24), in real time and interactively , the captured images, the results of the measurements made, documentation and reference images enriched by the audio and / or video and / or text comments, and a memory for recording the information and the captured images and the results of the measurements exchanged between the operator (20) and the remote experts (24).
    [0003]
    3. Device according to claim 2 wherein said portable housing (30) comprises a software module for generating an operating context based on location data provided by a GPS, weather data from the place of endoscopy and data on the communications network.
    [0004]
    4. Device according to claim 2 wherein said wireless communication interface comprises a conversion module captured images of an endoscope-specific data format (26) to a data format exploitable by the remote terminal (10). . 3033046 10
    [0005]
    The device of claim 1 wherein said portable housing (30) further includes a connectivity test module and a bandwidth calculation module for testing whether the network allows for reliable communication between the operator (20) and the experts (24).
    [0006]
    6. Device according to claim 5 wherein said portable housing (30) further comprises an encoder for adjusting the flow of information exchanged between the operator (20) and the experts (24) according to the characteristics of the network to maintain maximum image quality and minimal latency.
    [0007]
    7. Device according to claim 5 wherein said portable housing (30) comprises a connector for being connected to a headset, a tablet and / or a laptop. A method of assisting remote diagnosis during an aircraft engine condition check for collaborative analysis of images and measurements by an operator (20) by means of an endoscope (26) or camera and intended to be transmitted to at least one remote expert (24), the method comprising the steps of: - transmitting said images and said measurements of the endoscope (26) to a housing portable device (30) connectable to an operator-controlled tablet (20); - manually and / or automatically enrich said images and said measurements with audio and / or video and / or text comments, - exchange in real time and interactively, via a wireless connection, the images, the measurements, a documentation and enriched reference images between said portable housing (30) and at least one remote terminal (10) used by the expert (24) for analyzing said images and said measurements and for collaborative diagnosis. The method of claim 8 further comprising converting the results of the measurements made and the captured images of a data format specific to the endoscope to a data format exploitable by said portable housing (30) and by the remote terminal (10). A computer program stored on a recording medium and having instructions for performing the steps of the method according to one of claims 8 or 9 when executed on a computer or tablet.
    类似技术:
    公开号 | 公开日 | 专利标题
    JP6860617B2|2021-04-14|Systems and methods for performing data analysis workflows in non-destructive testing systems
    US10387237B2|2019-08-20|Systems and methods for analyzing data in a non-destructive testing system
    FR3033046B1|2019-06-14|METHOD AND DEVICE FOR CONTROLLING THE STATUS OF A REMOTE AIRCRAFT ENGINE
    JP6389192B2|2018-09-12|System and method for collaborating in a non-destructive inspection system
    CN104299287A|2015-01-21|Method and system for routing inspection of switching station
    CN108307150A|2018-07-20|A kind of visualized presence monitoring system and method having personnel's management and control function
    WO2014116423A2|2014-07-31|Systems and methods for sharing data in a non-destructive testing system
    EP2948901A2|2015-12-02|Systems and methods for collaborating in a non-destructive testing system using location information
    CN109657025B|2020-09-11|On-site investigation information collection system and on-site investigation management system
    US20180329904A1|2018-11-15|Intelligent and automated review of industrial asset integrity data
    US20150332355A1|2015-11-19|Asset estimate generation system
    US20120229631A1|2012-09-13|Vendor and scanner independent common workstation for security
    CN107547870A|2018-01-05|A kind of in-circuit diagnostic system and method
    WO2015155487A1|2015-10-15|Method and device for remote endoscopy of an aircraft engine
    KR102059444B1|2019-12-26|Real time remote system and method for observing tunnel working face
    WO2020232608A1|2020-11-26|Transmission and distribution device diagnosis method, apparatus, and system, computing device, medium, and product
    CN208873041U|2019-05-17|Security control platform
    WO2022031835A1|2022-02-10|Networked system for real-time computer-aided augmentation of live input video stream
    CN110719456A|2020-01-21|Software and hardware combined front-end equipment fault diagnosis method
    Stefanowicz2013|Mobile video streaming in equipment maintenance
    US20140036771A1|2014-02-06|Networked Video Bridge Device
    Plasson2006|CoRoTlog
    同族专利:
    公开号 | 公开日
    US10351263B2|2019-07-16|
    CN107249983A|2017-10-13|
    CA2976582A1|2016-09-01|
    FR3033046B1|2019-06-14|
    RU2017133020A|2019-03-25|
    EP3262392A1|2018-01-03|
    EP3262392B1|2020-10-14|
    RU2017133020A3|2019-07-30|
    BR112017017931A2|2018-04-10|
    US20180346150A1|2018-12-06|
    RU2707681C2|2019-11-28|
    CN107249983B|2021-04-20|
    WO2016135402A1|2016-09-01|
    引用文献:
    公开号 | 申请日 | 公开日 | 申请人 | 专利标题
    EP0725373A2|1995-02-01|1996-08-07|A.L.D. Advanced Logistices Developments Ltd.|A system and method for failure reporting and collection|
    WO2001071500A1|2000-03-22|2001-09-27|General Electric Company|Reduced access field inspection system and apparatus|
    JP2004024340A|2002-06-21|2004-01-29|Olympus Corp|Endoscope equipment|
    US20080109187A1|2006-11-02|2008-05-08|Kollgaard Jeffrey R|Remote nondestructive inspection systems and methods|
    US20140208163A1|2013-01-22|2014-07-24|General Electric Company|Systems and methods for analyzing data in a non-destructive testing system|
    RU2038991C1|1993-12-09|1995-07-09|Александр Сергеевич Шенгардт|Method of estimation and maintenance of reliability of aircraft and their power plants during operation of aviation materiel|
    US6850252B1|1999-10-05|2005-02-01|Steven M. Hoffberg|Intelligent electronic appliance system and method|
    US7350159B2|2001-05-08|2008-03-25|Snap-On Incorporated|Integrated diagnostic system|
    GB0211644D0|2002-05-21|2002-07-03|Wesby Philip B|System and method for remote asset management|
    FR2857094B1|2003-07-04|2005-08-26|Snecma Moteurs|DEVICE FOR SEARCHING AND DETECTING DEFECTS OF PARTS BY ENDOSCOPY|
    US8157205B2|2006-03-04|2012-04-17|Mcwhirk Bruce Kimberly|Multibody aircrane|
    US9262818B2|2007-05-01|2016-02-16|Pictometry International Corp.|System for detecting image abnormalities|
    FR2939924B1|2008-12-15|2012-10-12|Snecma|IDENTIFICATION OF FAILURES IN AN AIRCRAFT ENGINE|
    US8135989B2|2009-02-27|2012-03-13|Red Hat, Inc.|Systems and methods for interrogating diagnostic target using remotely loaded image|
    US20100246605A1|2009-03-31|2010-09-30|Motorola, Inc.|Enhanced visual experience for a participant in a streaming event|
    RU87816U1|2009-06-17|2009-10-20|Российская Федерация, От Имени Которой Выступает Министерство Промышленности И Торговли Российской Федерации|NETWORK SYSTEM FOR DIAGNOSTICS OF AIRCRAFT ENGINES|
    EP3416153A1|2011-05-03|2018-12-19|iOnRoad Technologies Ltd.|Parking space identifying method and system|
    CN102320382A|2011-07-07|2012-01-18|中国国际航空股份有限公司|Aircraft performance detection method|
    US8792705B2|2011-11-03|2014-07-29|United Technologies Corporation|System and method for automated defect detection utilizing prior data|
    JP2015531900A|2012-06-07|2015-11-05|ゾール メディカル コーポレイションＺＯＬＬ Ｍｅｄｉｃａｌ Ｃｏｒｐｏｒａｔｉｏｎ|System and method for video capture, user feedback, reporting, adaptive parameters, remote data access in vehicle safety monitoring|
    FR3005042B1|2013-04-26|2016-01-01|Snecma|WEAVING OR WINDING MACHINE FOR FIBROUS TEXTURE FOR ANOMALY CONTROL BY IMAGE ANALYSIS|
    FR3006373B1|2013-05-31|2015-05-22|Snecma|TURBOMACHINE HOUSING HAVING ENDOSCOPIC ORIFICE|
    US20140357993A1|2013-05-31|2014-12-04|eagleyemed, Inc.|Dynamic adjustment of image compression for high resolution live medical image sharing|
    US9053516B2|2013-07-15|2015-06-09|Jeffrey Stempora|Risk assessment using portable devices|
    FR3019898B1|2014-04-11|2018-06-01|Safran Aircraft Engines|METHOD AND DEVICE FOR ENDOSCOPY OF A REMOTE AIRCRAFT ENGINE|JP6763206B2|2016-06-17|2020-09-30|株式会社Ｊｖｃケンウッド|Imaging device, operation method of imaging device, and imaging program|
    RU2703850C1|2018-05-29|2019-10-22|Федеральное государственное бюджетное учреждение науки Сибирский федеральный научный центр агробиотехнологий Российской академии наук |Method for remote diagnosis of internal combustion engine technical state|
    US11126170B2|2020-02-11|2021-09-21|Mistras Group, Inc.|System and method for remote structural health monitoring|
    法律状态:
    2016-02-15| PLFP| Fee payment|Year of fee payment: 2 |
    2016-08-26| PLSC| Publication of the preliminary search report|Effective date: 20160826 |
    2017-02-13| PLFP| Fee payment|Year of fee payment: 3 |
    2018-01-23| PLFP| Fee payment|Year of fee payment: 4 |
    2018-02-02| CD| Change of name or company name|Owner name: SAFRAN AIRCRAFT ENGINES, FR Effective date: 20170719 |
    2020-01-22| PLFP| Fee payment|Year of fee payment: 6 |
    2021-01-20| PLFP| Fee payment|Year of fee payment: 7 |
    2022-01-19| PLFP| Fee payment|Year of fee payment: 8 |
    优先权:
    申请号 | 申请日 | 专利标题
    FR1551523A|FR3033046B1|2015-02-23|2015-02-23|METHOD AND DEVICE FOR CONTROLLING THE STATUS OF A REMOTE AIRCRAFT ENGINE|
    FR1551523|2015-02-23|FR1551523A| FR3033046B1|2015-02-23|2015-02-23|METHOD AND DEVICE FOR CONTROLLING THE STATUS OF A REMOTE AIRCRAFT ENGINE|
    CN201680011376.XA| CN107249983B|2015-02-23|2016-02-22|Method and device for remotely checking the state of an aircraft engine|
    EP16713523.5A| EP3262392B1|2015-02-23|2016-02-22|Method and device for remotely inspecting the state of an aircraft engine|
    BR112017017931-8A| BR112017017931A2|2015-02-23|2016-02-22|remote diagnostic assistance device, remote diagnostic assistance method, and computer program product|
    CA2976582A| CA2976582A1|2015-02-23|2016-02-22|Method and device for remotely inspecting the state of an aircraft engine|
    PCT/FR2016/050401| WO2016135402A1|2015-02-23|2016-02-22|Method and device for remotely inspecting the state of an aircraft engine|
    RU2017133020A| RU2707681C2|2015-02-23|2016-02-22|Method and device for remote inspection of aircraft engine condition|
    US15/552,634| US10351263B2|2015-02-23|2016-02-22|Method and device for remotely inspecting the state of an aircraft engine|
    [返回顶部]